Information processing apparatus, storage medium having stored therein information processing program, information processing system, and information processing method

ABSTRACT

A portable information processing apparatus includes a display, stereo loudspeakers, a pair of vibrators, and one or more processors. The pair of vibrators is provided on a left side and a right side of a main body of the information processing apparatus with respect to a center of the display. The one or more processors are configured to: generate sound signals to be output to the stereo loudspeakers, and output the sound signals; generate vibration signals for vibrating the pair of vibrators by vibrations, each having a plurality of frequency components, and output the vibration signals; and control an image to be displayed on the display.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of Ser. No. 15/799,374 filed on Oct.31, 2017; which is a continuation of U.S. patent application Ser. No.14/581,561, filed on Dec. 23, 2014, now U.S. Pat. No. 9,833,702; whichclaims priority to Japanese Patent Application No. 2014-15138, filed onJan. 30, 2014. The disclosures of these applications are incorporatedherein by reference.

FIELD

The technology shown here relates to an information processingapparatus, a storage medium having stored therein an informationprocessing program, an information processing system, and an informationprocessing method, and in particular, relates to an informationprocessing apparatus, an information processing system, and aninformation processing method for, for example, controlling vibrationsto be imparted to a user, and a storage medium having stored therein aninformation processing program for, for example, controlling vibrationsto be imparted to a user.

BACKGROUND AND SUMMARY

Conventionally, a game apparatus for imparting vibrations to the mainbody of the apparatus is known. For example, the game apparatus impartsvibrations to the main body of the apparatus in accordance with the gamesituation and transmits the vibrations to the fingers and the hands of auser holding the apparatus.

In the game apparatus, however, the vibrations to be imparted to thefingers and the hands of the user are poor in variety.

Therefore, it is an object of an exemplary embodiment to provide aninformation processing apparatus, an information processing system, andan information processing method that are capable of impartingvibrations rich in variety to a user, and a storage medium having storedtherein an information processing program capable of impartingvibrations rich in variety to a user.

To achieve the above object, the exemplary embodiment can employ, forexample, the following configurations. It should be noted that it isunderstood that, to interpret the descriptions of the claims, the scopeof the claims should be interpreted only by the descriptions of theclaims. If there is a conflict between the descriptions of the claimsand the descriptions of the specification, the descriptions of theclaims take precedence.

An exemplary configuration of an information processing apparatusaccording to an exemplary embodiment is a portable informationprocessing apparatus. The information processing apparatus includes adisplay, stereo loudspeakers, a pair of vibrators, and one or moreprocessors. The pair of vibrators is provided on a left side and a rightside of a main body of the information processing apparatus with respectto a center of the display. The one or more processors are configuredto: generate sound signals to be output to the stereo loudspeakers, andoutput the sound signals; generate vibration signals for vibrating thepair of vibrators by vibrations, each having a plurality of frequencycomponents, and output the vibration signals; and control an image to bedisplayed on the display.

Based on the above, a pair of vibrators provided on the left side andthe right side imparts vibrations, each having a plurality of frequencycomponents. This makes it possible to impart vibrations rich in varietyto a user.

Further, in the generation and output of the vibration signals, thevibration signals for vibrating the vibrators by vibrations, each in afrequency range having a predetermined width, may be generated.

Based on the above, the pair of vibrators imparts vibrations, each in afrequency range having a predetermined width (e.g., vibrations in whichvalues equal to or greater than a predetermined magnitude continue in atleast one frequency range having a predetermined width). This makes itpossible to impart vibrations rich in variety to the user.

Further, the vibration signals generated in the generation and output ofthe vibration signals may be subjected to predetermined processing andthereby may be able to be decoded into signals, each having a pluralityof frequency components.

Based on the above, vibration signals are generated, whereby it ispossible to directly control the frequency components of the vibrationsto be imparted.

Further, in the generation and output of the vibration signals, a firstvibration signal for vibrating one of the pair of vibrators and a secondvibration signal for vibrating the other of the pair of vibrators may begenerated and output.

Based on the above, vibration signals are output to the pair ofvibrators. This makes it possible to independently control thevibrators.

Further, the stereo loudspeakers may be provided on the left side andthe right side of the main body of the information processing apparatuswith respect to the center of the display. In this case, the stereoloudspeakers and the vibrators may be placed at different positions inthe main body of the information processing apparatus.

Based on the above, loudspeakers are also provided on the left side andthe right side of the main body of the information processing apparatussimilarly to the vibrators. This makes it easy to independently controlvibrations and sounds while performing control by combining thevibrations and the sounds.

Further, the stereo loudspeakers and the vibrators may be placed atpositions close to each other in the main body of the informationprocessing apparatus.

Based on the above, the vibrators and the loudspeakers are placed closeto each other. This makes it easy to independently control vibrationsand sounds while performing control by combining the vibrations and thesounds.

Further, the information processing apparatus may further include ahousing to be held by a user with both hands. In this case, one of thepair of vibrators may be placed in a portion of the housing that is heldby the user with a right hand, and the other of the pair of vibratorsmay be placed in a portion of the housing that is held by the user witha left hand.

Based on the above, it is possible to efficiently transmit vibrations tothe hands of the user holding a housing of the information processingapparatus.

Further, the information processing apparatus may further include aconversion unit. The conversion unit converts the vibration signals intoanalog signals, drives the vibrators using the analog signals, andvibrates the vibrators by vibrations, each having a plurality offrequency components.

Based on the above, vibration signals are converted into analog signals.This makes it possible to easily drive the vibrators.

Further, in the generation and output of the vibration signals, thevibration signals may be generated to obtain vibrations corresponding tothe image controlled in the control of the image.

Based on the above, vibrations corresponding to a display image areimparted to the user. This makes it possible to provide a highlyrealistic experience to the user.

Further, in the control of the image, at least one object may bedisplayed on the display. In the generation and output of the vibrationsignals, a position of a vibration source to be perceived based on thevibrations of the pair of vibrators may be controlled in accordance witha position of the object displayed on the display.

Based on the above, an object display position is a vibration source,whereby it is possible to provide a more highly realistic experience tothe user.

Further, in the generation and output of the sound signals, the soundsignals may be generated to obtain sounds corresponding to the imagecontrolled in the control of the image.

Based on the above, sounds corresponding to a display image areimparted. This makes it possible to provide a highly realisticexperience to the user.

Further, in the control of the image, at least one object may bedisplayed on the display. In the generation and output of the soundsignals, a position where sound sources are localized for sounds outputfrom the stereo loudspeakers may be controlled in accordance with aposition of the object displayed on the display.

Based on the above, an object display position is a sound sourcelocalization position, whereby it is possible to provide a more highlyrealistic experience to the user.

Further, in the generation and output of the vibration signals, thevibration signals may be generated to obtain vibrations corresponding tosounds controlled in the generation and output of the sound signals.

Based on the above, vibrations corresponding to sounds are imparted tothe user. This makes it possible to provide a highly realisticexperience to the user.

Further, in the generation and output of the sound signals, sounds to beoutput from the stereo loudspeakers may be controlled so that soundsources are localized at a predetermined position. In the generation andoutput of the vibration signals, a position of a vibration source to beperceived based on the vibrations of the vibrators may be controlled inaccordance with the position of localization of the sound sources.

Based on the above, a sound source localization position is a vibrationsource. This makes it possible to provide a more highly realisticexperience to the user.

Further, the information processing apparatus may further include aprogram storage medium. The program storage medium stores a soundgeneration program for generating the sound signals and a vibrationgeneration program for generating the vibration signals. In this case,in the generation and output of the sound signals, the sound signals maybe generated by executing the sound generation program. In thegeneration and output of the vibration signals, the vibration signalsmay be generated by executing the vibration generation program.

Based on the above, sounds and vibrations are controlled by executingprograms. This enables flexible control.

Further, the information processing apparatus may further include a datastorage medium. The data storage medium stores sound data regarding thesound signals and vibration data regarding the vibration signals. Inthis case, in the generation and output of the sound signals, the sounddata stored in the data storage medium may be read to generate the soundsignals. In the generation and output of the vibration signals, thevibration data stored in the data storage medium may be read to generatethe vibration signals.

Based on the above, vibration data and sound data prepared in advanceare used. This makes it possible to present vibrations and sounds richin variety.

Further, in the control of the image, at least one object moving from acorresponding position in a virtual world displayed on the display to acorresponding position in the virtual world outside the display areaand/or moving from the corresponding position in the virtual worldoutside the display area to the corresponding position in the virtualworld displayed on the display may be displayed on the display. In thegeneration and output of the vibration signals, even if the object isplaced at the corresponding position in the virtual world outside thedisplay area, a position of a vibration source to be perceived based onthe vibrations of the vibrators may be controlled in accordance with theplacement position of the object.

Based on the above, even if an object is placed at a correspondingposition in a virtual world outside a display area, the user is causedto perceive a vibration source and thereby can imagine the object placedin the corresponding position in the virtual world outside the displayarea.

Further, in the generation and output of the sound signals, even if theobject is placed at the corresponding position in the virtual worldoutside the display area, a position where sound sources are localizedfor sounds output from the stereo loudspeakers may be controlled inaccordance with the placement position of the object.

Based on the above, even if the object is placed at the correspondingposition in the virtual world outside the display area, the positionwhere sound sources are localized is outside the display screen. Thismakes it easier for the user to recognize the object placed at thecorresponding position in the virtual world outside the display area.

Further, in the generation and output of the vibration signals, even ifthe object is placed at the corresponding position in the virtual worldoutside the display area, the pair of vibrators may be simultaneouslyvibrated based on distribution of intensities of vibrationscorresponding to the placement position of the object, thereby causing auser to perceive the placement position as the vibration source based onthe vibrations of the vibrators.

Based on the above, the distribution of intensities of vibrations to beimparted to the pair of vibrators is adjusted. Thus, even in a spaceformed outside between the pair of vibrators, it is possible to easilyset the vibration source.

Further, in the generation and output of the vibration signals, if theobject is placed at the corresponding position in the virtual worlddisplayed on the display, the position of the vibration source may becontrolled in accordance with the placement position of the object.

Based on the above, even if the object is placed at a correspondingposition in the virtual world displayed on a display, it is possible tocause the user to perceive the object placed at the correspondingposition as the vibration source.

Further, in the generation and output of the vibration signals, at leasteither if the object moves from the corresponding position in thevirtual world displayed on the display to the corresponding position inthe virtual world outside the display area, or if the object moves fromthe corresponding position in the virtual world outside the display areato the corresponding position in the virtual world displayed on thedisplay, the position of the vibration source may be controlled withoutinterruption.

Based on the above, if the object moves from the virtual world withinthe display area to the virtual world outside the display area, and ifthe object moves from the virtual world outside the display area to thevirtual world within the display area, it is possible to cause the userto perceive the object as the vibration source without interruption.

Further, in the generation and output of the vibration signals, thevibration signals may be generated such that as the object moves towarda position, in the virtual world, corresponding to a left referenceposition provided on the left side of the main body of the informationprocessing apparatus, vibrations of the vibrators corresponding to theleft side increase, and as the object moves away from the position, inthe virtual world, corresponding to the left reference position, thevibrations of the vibrators corresponding to the left side decrease. Inthe generation and output of the vibration signals, the vibrationsignals may be generated such that as the object moves toward aposition, in the virtual world, corresponding to a right referenceposition provided on the right side of the main body of the informationprocessing apparatus, vibrations of the vibrators corresponding to theright side increase, and as the object moves away from the position, inthe virtual world, corresponding to the right reference position, thevibrations of the vibrators corresponding to the right side decrease.

Based on the above, if the object is moving at a position in the virtualworld corresponding to a reference position provided to the left andright of the main body of the information processing apparatus,vibrations to be imparted by the vibrators and corresponding to thereference position are maximized. This makes it possible to cause theuser to perceive the moving object as the vibration source with morecertainty.

Further, in the generation and output of the vibration signals, if theobject has moved to pass through a position, in the virtual world,corresponding to a left reference position provided on the left side ofthe main body of the information processing apparatus, the vibrationsignals may be generated such that magnitudes of vibrations of thevibrators corresponding to the left reference position increase inaccordance with the movement of the object until the object reaches theposition, in the virtual world, corresponding to the left referenceposition, and after the object has passed through the position, in thevirtual world, corresponding to the left reference position, thevibration signals may be generated such that the magnitudes of thevibrations decrease in accordance with the movement of the object. Inthe generation and output of the vibration signals, if the object hasmoved to pass through a position, in the virtual world, corresponding toa right reference position provided on the right side of the main bodyof the information processing apparatus, the vibration signals may begenerated such that magnitudes of vibrations of the vibratorscorresponding to the right reference position increase in accordancewith the movement of the object until the object reaches the position,in the virtual world, corresponding to the right reference position, andafter the object has passed through the position, in the virtual world,corresponding to the right reference position, the vibration signals maybe generated such that the magnitudes of the vibrations decrease inaccordance with the movement of the object.

Based on the above, if the object passes through a position in thevirtual world corresponding to a reference position provided to the leftand right of the main body of the information processing apparatus,vibrations to be imparted by the vibrators and corresponding to thereference position are maximized. This makes it possible to cause theuser to perceive the moving object as the vibration source with morecertainty.

Further, the exemplary embodiment may be carried out in the form of anon-transitory computer-readable storage medium having stored thereinhaving an information processing program, an information processingsystem, or an information processing method.

According to the exemplary embodiment, a pair of vibrators provided onthe left side and the right side imparts vibrations, each having aplurality of frequency components. This makes it possible to impartvibrations rich in variety to a user.

These and other objects, features, aspects and advantages of theexemplary embodiments will become more apparent from the followingdetailed description of the exemplary embodiments when taken inconjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a plan view of a non-limiting example of the externalappearance of an information processing apparatus 3;

FIG. 2 is a block diagram showing a non-limiting example of theconfiguration of the information processing apparatus 3;

FIG. 3 is a block diagram showing a non-limiting example of theconfiguration of a vibration generation section 37;

FIG. 4 is a diagram showing a non-limiting example where the main bodyof the information processing apparatus 3 vibrates and simultaneously,sounds are output, in accordance with the display position of a virtualobject OBJ displayed on a display screen of a display section 35;

FIG. 5 is diagrams showing non-limiting examples of images of a firstexemplary game displayed on the display screen of the display section35;

FIG. 6 is a diagram illustrating non-limiting examples of vibrationsimparted to the main body of the information processing apparatus 3 andnon-limiting examples of sounds output from the information processingapparatus 3 in the first exemplary game;

FIG. 7 is diagrams illustrating non-limiting examples of the spectra ofvibrations imparted to the main body of the information processingapparatus 3 and non-limiting examples of the spectra of sounds outputfrom the information processing apparatus 3, near a time T1 in the firstexemplary game;

FIG. 8 is diagrams showing a non-limiting example where, when thevirtual object OBJ moves from within the display screen of the displaysection 35 to outside the display screen in a second exemplary game, themain body of the information processing apparatus 3 vibrates andsimultaneously, sounds are output;

FIG. 9 is a diagram showing non-limiting examples of the intensities ofvibrations imparted to the main body of the information processingapparatus 3 when the virtual object OBJ moves from within the displayscreen of the display section 35 to outside the display screen in thesecond exemplary game;

FIG. 10 is a diagram showing non-limiting examples of main data andprograms stored in a storage section 32 of the information processingapparatus 3; and

FIG. 11 is a flow chart showing a non-limiting example of gameprocessing performed by the information processing apparatus 3.

DETAILED DESCRIPTION OF NON-LIMITING EXAMPLE EMBODIMENTS

With reference to the drawings, an information processing apparatus forexecuting an information processing program according to an exemplaryembodiment is described. While the information processing programaccording to the exemplary embodiment can be applied by being executedby any computer system, a handheld information processing apparatus 3 (atablet terminal) is used as an example of the information processingapparatus, and the information processing program according to theexemplary embodiment is described using an information processingprogram executed by the information processing apparatus 3. For example,the information processing apparatus 3 can execute a program and apre-installed program (e.g., a game program) stored in a storage mediumsuch as an exchangeable optical disk or an exchangeable memory card, orreceived from another apparatus. As an example, the informationprocessing apparatus 3 can display on a screen an image generated by acomputer graphics process, such as a virtual space image viewed from avirtual camera set in a virtual space. The information processingapparatus 3 may be a device such as a general personal computer, astationary game apparatus, a mobile phone, a handheld game apparatus, ora PDA (Personal Digital Assistant). It should be noted that FIG. 1 is aplan view of an example of the external appearance of the informationprocessing apparatus 3.

In FIG. 1, the information processing apparatus 3 includes a displaysection 35, a sound output section 36, and an actuator 373. As anexample, the display section 35 is provided on the front surface of themain body of the information processing apparatus 3. For example, thedisplay section 35 includes an LCD (Liquid Crystal Display) and mayemploy, for example, a display device using EL. Further, the displaysection 35 may be a display device capable of displaying astereoscopically viewable image.

A touch panel 341, which is an example of an input section 34, isprovided so as to cover a display screen of the display section 35. Thetouch panel 341 detects the position of an input provided to apredetermined input surface (e.g., the display screen of the displaysection 35). It should be noted that the input section 34 is an inputdevice that allows a user of the information processing apparatus 3 toinput an operation, and the input section 34 may be any input device.For example, as the input section 34, an operation section such as aslide pad, an analog stick, a directional pad, an operation button, orthe like may be provided on the side surfaces, the back surface, or thelike of the main body of the information processing apparatus 3.Further, the input section 34 may be a sensor for detecting theorientation and the motion of the main body of the informationprocessing apparatus 3. For example, the input section 34 may be anacceleration sensor for detecting the acceleration generated in the mainbody of the information processing apparatus 3, an angular velocitysensor (a gyro sensor) for detecting the amount of rotation of the mainbody of the information processing apparatus 3, or the like.

The sound output section 36 includes loudspeakers for outputting sounds,and in the example shown in FIG. 1, includes a pair of stereoloudspeakers (a left sound output section 36L and a right sound outputsection 36R) provided on the left and right of the upper side surface orthe back surface of the information processing apparatus 3. The soundoutput section 36 performs D/A conversion on sound signals (a left soundcontrol signal and a right sound control signal) output from a controlsection 31 described later, thereby generating analog sound signals (aleft analog sound signal and a right analog sound signal). Then, thesound output section 36 outputs the analog sound signals to theloudspeakers (e.g., the stereo loudspeakers), thereby outputting sounds.

The actuator 373 is a vibration actuator (a vibrator) for impartingpredetermined vibrations to the main body of the information processingapparatus 3 and is included in a vibration generation section 37described later. In the example shown in FIG. 1, the actuator 373includes a left actuator 373L, which is provided on the left and insidethe main body of the information processing apparatus 3, and a rightactuator 373R, which is provided on the right and inside the main bodyof the information processing apparatus 3. Specifically, as indicated bydashed line areas in FIG. 1, the left actuator 373L is provided on theleft side of the display section 35, which is a position near the lefthand of the user when holding a left end portion of the informationprocessing apparatus 3 with the left hand. Further, the right actuator373R is provided on the right side of the display section 35, which is aposition near the right hand of the user when holding a right endportion of the information processing apparatus 3 with the right hand.Further, the vibration generation section 37 performs D/A conversion onvibration control signals (a left vibration control signal and a rightvibration control signal) output from the control section 31 describedlater, thereby generating analog vibration signals (a left analogvibration signal and a right analog vibration signal). Then, thevibration generation section 37 outputs driving signals obtained byamplifying the analog vibration signals to the actuator 373 (the leftactuator 373L and the right actuator 373R), thereby driving the actuator373.

It should be noted that as is clear from FIG. 1, the display screen ofthe display section 35, the left sound output section 36L, and the rightsound output section 36R, which are provided in the informationprocessing apparatus 3, are placed at positions close to one another.The display screen of the display section 35, the left actuator 373L,and the right actuator 373R are placed at positions close to oneanother. Further, the left sound output section 36L and the leftactuator 373L are placed at positions close to each other, but aredifferent units disposed at different positions. The right sound outputsection 36R and the right actuator 373R are placed at positions close toeach other, but are different units disposed at different positions.Consequently, it is possible to include a unit dedicated to outputtingvibrations and a unit dedicated to outputting sounds. Thus, it ispossible to output vibrations and sounds more accurately than when ageneral-purpose unit is shared. It should be noted that modules intowhich a unit for outputting vibrations and a unit for outputting soundsare combined and integrated may be provided on the left and right of theinformation processing apparatus 3.

Next, with reference to FIG. 2, the internal configuration of theinformation processing apparatus 3 is described. It should be noted thatFIG. 2 is a block diagram showing an example of the configuration of theinformation processing apparatus 3.

In FIG. 2, the information processing apparatus 3 includes the controlsection 31, a storage section 32, and a program storage section 33 inaddition to the input section 34, the display section 35, the soundoutput section 36, and the vibration generation section 37, which aredescribed above. It should be noted that the information processingapparatus 3 may be composed of one or more apparatuses including aninformation processing apparatus having at least the control section 31,and another apparatus.

The control section 31 is information processing means (a computer) forperforming various types of information processing and is, for example,a CPU. The control section 31 has the function of performing, as thevarious types of information processing, processing or the likecorresponding to an operation performed on the input section 34 by theuser. For example, the functions of the control section 31 are achievedby, for example, the CPU executing a predetermined program.

As the various types of information processing, the control section 31controls the display of an image to be displayed on the display section35. Further, as the various types of information processing, the controlsection 31 outputs, to the sound output section 36, sound controlsignals (e.g., digital sound signals) for controlling sounds to beoutput from the stereo loudspeakers. Further, as the various types ofinformation processing, the control section 31 outputs, to the vibrationgeneration section 37, vibration control signals (e.g., digitalvibration signals) for controlling vibrations of the actuator 373 (theleft actuator 373L and the right actuator 373R).

The storage section 32 stores various data to be used when the controlsection 31 performs the above information processing. The storagesection 32 is, for example, a memory accessible by the CPU (the controlsection 31).

The program storage section 33 stores a program. The program storagesection 33 may be any storage device (storage medium) accessible by thecontrol section 31. For example, the program storage section 33 may be astorage device provided in the information processing apparatus 3 havingthe control section 31, or may be a storage medium detachably attachedto the information processing apparatus 3 having the control section 31.Alternatively, the program storage section 33 may be a storage device (aserver or the like) connected to the control section 31 via a network.The control section 31 (the CPU) may read a part or all of the programto the storage section 32 at appropriate timing and execute the readprogram.

Next, with reference to FIG. 3, the configuration of the vibrationgeneration section 37 is described. It should be noted that FIG. 3 is ablock diagram showing an example of the configuration of the vibrationgeneration section 37.

In FIG. 3, the vibration generation section 37 includes a codec section371, an amplification section 372, the left actuator (left vibrator)373L, and the right actuator (right vibrator) 373R.

The codec section 371 acquires vibration control signals output from thecontrol section 31 and performs a predetermined decoding process on thevibration control signals, thereby generating analog vibration signals.Then, the codec section 371 outputs the analog vibration signals to theamplification section 372. For example, if a plurality of actuators 373are provided and independent vibrations are generated by the respectiveactuators 373 (e.g., the left actuator 373L and the right actuator373R), the control section 31 outputs vibration control signals (e.g., aleft vibration control signal CSL and a right vibration control signalCSR) for controlling vibrations in the respective actuators 373. In thiscase, the codec section 371 decodes the vibration control signals outputfrom the control section 31, thereby generating analog vibration signals(e.g., a left analog vibration signal ASL and a right analog vibrationsignal ASR) for generating vibrations in the respective actuators 373.Then, the codec section 371 outputs the analog vibration signals to theamplification section 372.

The amplification section 372 amplifies the analog vibration signalsoutput from the codec section 371, thereby generating driving signalsfor driving the actuator 373. Then, the amplification section 372outputs the driving signals to the actuator 373. For example, if aplurality of actuators 373 are provided (e.g., if the left actuator 373Land the right actuator 373R are provided), the amplification section 372increases changes in the amplitudes of the current and/or the voltage ofeach of the analog vibration signals (e.g., the left analog vibrationsignal ASL and the right analog vibration signal ASR) output from thecodec section 371, thereby generating driving signals (e.g., a leftdriving signal DSL and a right driving signal DSR). Then, theamplification section 372 outputs the driving signals to the respectiveactuators 373 (e.g., the left actuator 373L and the right actuator373R).

The actuator 373 is driven in accordance with the driving signals outputfrom the amplification section 372, thereby imparting vibrationscorresponding to the driving signals to the main body of the informationprocessing apparatus 3. For example, as shown in FIG. 1, the actuator373 includes the left actuator 373L and the right actuator 373R, whichare provided on the left side and the right side of the main body of theinformation processing apparatus 3 with respect to the center of thedisplay screen of the display section 35. As an example, the leftactuator 373L is placed in a part of the main body of the informationprocessing apparatus 3 that is held by the user with the left hand, andthe right actuator 373R is placed in a part of the main body of theinformation processing apparatus 3 that is held by the user with theright hand. Here, the method of the actuator 373 to impart vibrations tothe main body of the information processing apparatus 3 may be anymethod. For example, the actuator 373 may use the method of generatingvibrations by an eccentric motor (ERM: Eccentric Rotating Mass), themethod of generating vibrations by a linear vibrator (LRA: LinearResonant Actuator), the method of generating vibrations by apiezoelectric element, or the like. If the driving signals to be outputfrom the amplification section 372 are generated in accordance with themethod of the actuator 373 to generate vibrations, an actuator using anymethod can impart various vibrations to the user of the informationprocessing apparatus 3.

It should be noted that in the configuration of the vibration generationsection 37, one or more codec sections and one or more amplificationsections may be provided. For example, a codec section and anamplification section can be provided in each of a plurality ofactuators 373 that are provided. In the above description, an examplehas been used where driving signals for driving the actuator 373 aregenerated by amplifying the analog vibration signals generated by thecodec section 371. Alternatively, the signals output from the codecsection 371 to the amplification section 372 may be digital signals. Forexample, if the actuator 373 is driven by pulse width modulation (PWM)control, the codec section 371 may generate pulse signals for turning onand off the actuator 373. In this case, the signals output from thecodec section 371 to the amplification section 372 are digital vibrationsignals for controlling the driving of the actuator 373 using pulsewaves. Consequently, the amplification section 372 amplifies the digitalvibration signals.

Next, with reference to FIGS. 4 to 9, a description is given of anoverview of the processing performed by the information processingapparatus 3, before the description of specific processing performed bythe information processing apparatus 3. In the following descriptions,the process of performing a game where a player character PC and avirtual object OBJ move within or outside the display screen of thedisplay section 35 is used as an example of the information processingperformed by the information processing apparatus 3. It should be notedthat FIG. 4 is a diagram showing an example where the main body of theinformation processing apparatus 3 vibrates and simultaneously, soundsare output, in accordance with the display position of the virtualobject OBJ displayed on the display screen of the display section 35.FIG. 5 is diagrams showing images of a first exemplary game displayed onthe display screen of the display section 35. FIG. 6 is a diagramillustrating examples of vibrations imparted to the main body of theinformation processing apparatus 3 and examples of sounds output fromthe information processing apparatus 3 in the first exemplary game. FIG.7 is diagrams illustrating examples of the spectra of vibrationsimparted to the main body of the information processing apparatus 3 andexamples of the spectra of sounds output from the information processingapparatus 3, near a time T1 in the first exemplary game. FIG. 8 isdiagrams showing an example where, when the virtual object OBJ movesfrom within the display screen of the display section 35 to outside thedisplay screen in a second exemplary game, the main body of theinformation processing apparatus 3 vibrates and simultaneously, soundsare output. FIG. 9 is a diagram showing examples of the intensities ofvibrations imparted to the main body of the information processingapparatus 3 when the virtual object OBJ moves from within the displayscreen of the display section 35 to outside the display screen in thesecond exemplary game.

In the example shown in FIG. 4, a virtual object OBJ moving in a virtualworld is displayed on the display screen of the display section 35. Thevirtual object OBJ is displayed on the display screen of the displaysection 35 so as to move in the virtual world in accordance with anoperation of the user or automatically.

In accordance with the movement of the virtual object OBJ, the main bodyof the information processing apparatus 3 vibrates and simultaneously,sounds are output. For example, in accordance with the display positionof the virtual object OBJ displayed on the display screen of the displaysection 35, the left actuator 373L and the right actuator 373R, whichare provided in the main body of the information processing apparatus 3,generate vibrations to occur when the virtual object OBJ moves. As anexample, with the use of phantom sensation that stimulates two differentpoints in the user's skin (specifically, the left hand and the righthand of the user holding the main body of the information processingapparatus 3) to cause the user to perceive a pseudo stimulus at onepoint, the left actuator 373L and the right actuator 373R impart, to theuser of the information processing apparatus 3, vibrations for causingthe user to perceive the display position of the virtual object OBJ asthe vibration source in a pseudo manner (vibrations for causing thedisplay position of the virtual object OBJ to be the position where apseudo force sense is presented). Further, the pair of stereoloudspeakers (the left sound output section 36L and the right soundoutput section 36R) provided in the main body of the informationprocessing apparatus 3 output sounds such that the position where thesound sources are localized is the display position of the virtualobject OBJ displayed on the display screen of the display section 35. Asdescribed above, the display position of the virtual object OBJ, theposition to be perceived as the vibration source in a pseudo manner (apseudo force sense presentation position), and the sound sourcelocalization position are substantially matched. This can provide arealistic experience using visual sensation, tactile sensation, andauditory sense to the user. Further, vibrations imparted by the leftactuator 373L and the right actuator 373R and sounds output from theleft sound output section 36L and the right sound output section 36R aregenerated by imitating the vibrations and sounds when the virtual objectOBJ moves. This can further enhance the reality.

Next, with reference to FIG. 5, a description is given of a firstexemplary game, which can be played using the information processingapparatus 3. In the first exemplary game, in accordance with anoperation of the user of the information processing apparatus 3, atleast a part of a virtual world including a player character PC, whichcan move in the virtual world, is displayed on the display screen of thedisplay section 35. For example, the player character PC moves in thevirtual world in accordance with an operation of the user using theinput section 34 (e.g., the state of a time T0 shown in (A) of FIG. 5).

In the virtual world displayed on the display screen of the displaysection 35, a plurality of virtual objects OBJ1 to OBJ3 are placed. Ifthe virtual objects OBJ1 to OBJ3 have come into contact with the playercharacter PC in the virtual world (e.g., have been stepped on by theplayer character PC), then in the progression of the game, the sounds ofcontact (e.g., crushing sounds) are output from the informationprocessing apparatus 3, and simultaneously, the vibrations when thevirtual objects OBJ1 to OBJ3 have come into contact with the playercharacter PC are imparted to the information processing apparatus 3. Forexample, at a time T1, which is chronologically later than the time T0,the state where the player character PC has come into contact with thevirtual object OBJ1 in the virtual world is displayed on the displayscreen of the display section 35 (see (B) of FIG. 5). At this time, inthe first exemplary game, sounds indicating that the virtual object OBJ1and the player character PC have come into contact with each other areoutput from the stereo loudspeakers (the left sound output section 36Land the right sound output section 36R) such that the position where thecontact is displayed is the sound source localization position. Further,in the first exemplary game, vibrations indicating that the virtualobject OBJ1 and the player character PC have come into contact with eachother are imparted by the left actuator 373L and the right actuator 373Rsuch that the position where the contact is displayed is the position tobe perceived as the vibration source in a pseudo manner.

Further, at a time T2, which is chronologically later than the time T1,the state where the player character PC has come into contact with thevirtual object OBJ2 in the virtual world is displayed on the displayscreen of the display section 35 (see (C) of FIG. 5). At this time, inthe first exemplary game, sounds indicating that the virtual object OBJ2and the player character PC have come into contact with each other areoutput from the stereo loudspeakers such that the position where thecontact is displayed is the sound source localization position.Simultaneously, vibrations indicating that the virtual object OBJ2 andthe player character PC have come into contact with each other areimparted by the left actuator 373L and the right actuator 373R such thatthe position where the contact is displayed is the position to beperceived as the vibration source in a pseudo manner.

Further, at a time T3, which is chronologically later than the time T2,the state where the player character PC has come into contact with thevirtual object OBJ3 in the virtual world is displayed on the displayscreen of the display section 35 (see (D) of FIG. 5). At this time, inthe first exemplary game, sounds indicating that the virtual object OBJ3and the player character PC have come into contact with each other areoutput from the stereo loudspeakers such that the position where thecontact is displayed is the sound source localization position.Simultaneously, vibrations indicating that the virtual object OBJ3 andthe player character PC have come into contact with each other areimparted by the left actuator 373L and the right actuator 373R such thatthe position where the contact is displayed is the position to beperceived as the vibration source in a pseudo manner.

As shown in FIG. 6, in the first exemplary game, vibrations differentfrom each other are imparted by the left actuator 373L and the rightactuator 373R to the main body of the information processing apparatus3, thereby achieving the above phantom sensation. For example, at thetime T1, the amplitude of the vibration imparted by the left actuator373L to the main body of the information processing apparatus 3 (thevibration signal (the left side); for example, indicating the drivingsignal for driving the left actuator 373L or the left analog vibrationsignal) is greater than the amplitude of the vibration imparted by theright actuator 373R to the main body of the information processingapparatus 3 (the vibration signal (the right side); for example,indicating the driving signal for driving the right actuator 373R or theright analog vibration signal). This is because, as shown in (B) of FIG.5, at the time T1, on the display screen of the display section 35, theplayer character PC has come into contact with the virtual object OBJ1in an area to the left of the center of the display screen. Thus, thevibration to be imparted from the left side is made stronger than thevibration to be imparted from the right side, whereby it is possible tocause the user to perceive as the vibration source the position on theleft of the display screen where the contact is displayed.

On the other hand, at the time T3, the amplitude of the vibrationimparted by the right actuator 373R to the main body of the informationprocessing apparatus 3 is greater than the amplitude of the vibrationimparted by the left actuator 373L to the main body of the informationprocessing apparatus 3. This is because, as shown in (D) of FIG. 5, atthe time T3, on the display screen of the display section 35, the playercharacter PC has come into contact with the virtual object OBJ3 in anarea to the right of the center of the display screen. Thus, thevibration to be imparted from the right side is made stronger than thevibration to be imparted from the left side, whereby it is possible tocause the user to perceive as the vibration source the position on theright of display screen where the contact is displayed.

Further, as shown in FIG. 7, in the first exemplary game, each of theleft actuator 373L and the right actuator 373R imparts a vibrationhaving a plurality of frequency components (a vibration having not onlya single frequency component) to the main body of the informationprocessing apparatus 3. For example, (A) of FIG. 7 and (B) of FIG. 7show examples of the spectra of the vibrations imparted to the main bodyof the information processing apparatus 3 near the time T1 and indicatethat a vibration in a frequency range having a predetermined width (avibration in a wide band) is imparted by each of the left actuator 373Land the right actuator 373R to the main body of the informationprocessing apparatus 3. More specifically, a vibration having power inthe entire area of the frequency range from a frequency component lowerthan 10 Hz (hertz) to a frequency component higher than 1 kHz(kilohertz) is imparted by each of the left actuator 373L and the rightactuator 373R to the main body of the information processing apparatus3. Here, the sensory receptors of a human being for receiving cutaneoussensation include Merkel discs, Meissner corpuscles, Paciniancorpuscles, Ruffini endings, and the like. The Merkel discs are regardedas responding to vibrations at 0 to 200 Hz. The Meissner corpuscles areregarded as responding to vibrations at 20 to 100 Hz and regarded ashaving the highest sensitivity to vibrations near 30 Hz. Paciniancorpuscles are regarded as responding to vibrations at 100 to 300 Hz andregarded as having the highest sensitivity to vibrations near 200 Hz.The vibrations imparted by the left actuator 373L and the right actuator373R to the main body of the information processing apparatus 3 includevibrations in the frequency range of 0 to 1 kHz, which is regarded asbeing able to be felt by a human being or include part of vibrations inthis frequency range, and also include frequency components to whichthese sensory receptors can respond (frequency components in areas A toC shown in (A) of FIG. 7 and (B) of FIG. 7). This makes it possible toreproduce a touch rich in reality and present the touch to the user.

Further, as shown in FIG. 7, vibrations that can be imparted to the userof the information processing apparatus 3 and sounds that are outputfrom the information processing apparatus 3 can have different spectra.For example, the information processing apparatus 3 can impart, to theuser, vibrations around a wide band of 0 to 1000 Hz, which is regardedas being able to be felt by a human being, and can also output soundsaround a wide band of 20 to 20000 Hz, which include audio frequenciesaudible to the human ear. Thus, the vibrations and the sounds can havedifferent spectra by controlling different vibrators (the left actuator373L, the right actuator 373R, and the stereo loudspeakers) by differentcontrol signals.

In the exemplary embodiment, vibration data for imparting vibrations tothe main body of the information processing apparatus 3 and sound datafor outputting sounds from the information processing apparatus 3 may beprepared separately in advance. In this case, vibration datacorresponding to the type of vibrations to be imparted to the main bodyof the information processing apparatus 3 is extracted and read from theprepared vibration data, thereby generating vibration control signals.Further, sound data corresponding to sounds to be output from theinformation processing apparatus 3 is extracted and read from theprepared sound data, thereby generating sound control signals. It shouldbe noted that in the vibration data, vibration data for imparting avibration from the left actuator 373L and vibration data for imparting avibration from the right actuator 373R may be prepared separately. As anexample, a pair of left and right pieces of vibration data may beprepared in advance based on the position of the vibration source. Then,when vibrations are imparted to the main body of the informationprocessing apparatus 3, a pair of left and right pieces of vibrationdata corresponding to the position to be perceived as the vibrationsource may be read. Further, it goes without saying that also in thesound data, sound data for outputting a sound from the left loudspeakerand sound data for outputting a sound from the right loudspeaker may beprepared separately. Further, sound data prepared in advance may be usedas vibration data. Sound data is also data used to vibrate and drive adiaphragm of a loudspeaker and therefore can be used as data forvibrating and driving a vibrator (i.e., vibration data).

Further, in the exemplary embodiment, a vibration control signal (theleft vibration control signal CSL) for driving the left actuator 373Land a vibration control signal (the right vibration control signal CSR)for driving the right actuator 373R may be generated independently ofeach other, or may be generated by processing a single vibration controlsignal. For example, in the second case, a single vibration controlsignal prepared in advance in accordance with the intensity of thevibration for vibrating each actuator can be processed, therebygenerating the left vibration control signal CSL and the right vibrationcontrol signal CSR.

Next, with reference to FIG. 8, a description is given of a secondexemplary game, which can be played using the information processingapparatus 3. In the second exemplary game, a virtual object OBJ that canmove is placed in a virtual world, and the virtual object OBJ moves notonly in the virtual world displayed on the display screen of the displaysection 35, but also in the virtual world outside the display screen.

For example, as shown in (A) of FIG. 8, if the state where the virtualobject OBJ is moving in the virtual world displayed within the displayscreen of the display section 35 is displayed, then based on the displayposition of the virtual object OBJ, the sounds of the virtual object OBJmoving are output from the information processing apparatus 3, andsimultaneously, the vibrations of the virtual object OBJ moving areimparted to the information processing apparatus 3. Specifically, in thesecond exemplary game, sounds indicating that the virtual object OBJ ismoving within the display screen are output from the stereo loudspeakers(the left sound output section 36L and the right sound output section36R) such that the display position of the virtual object OBJ is thesound source localization position. Further, in the second exemplarygame, vibrations indicating that the virtual object OBJ is moving withinthe display screen are imparted by the left actuator 373L and the rightactuator 373R such that the display position of the virtual object OBJis the position to be perceived as the vibration source in a pseudomanner. Then, if the virtual object OBJ moves from the virtual worlddisplayed within the display screen of the display section 35 to thevirtual world outside the display screen, also the sound sourcelocalization position and the position to be perceived as the vibrationsource move to outside the display screen of the display section 35 inaccordance with the movement of the virtual object OBJ (see (B) of FIG.8).

Then, as shown in (C) of FIG. 8, if the virtual object OBJ moves in thevirtual world that is not displayed on the display screen of the displaysection 35, a position (the position of the virtual object OBJ indicatedby a dashed line in (C) of FIG. 8) at which the virtual object OBJ isassumed to move on the basis of a position in the virtual worlddisplayed on the display screen is set as the sound source localizationposition and the position to be perceived as the vibration source. Forexample, 3D audio effects are generated using surround technology,binaurally recorded sounds, and the like, thereby outputting, from thestereo loudspeakers, sounds indicating that the virtual object OBJ ismoving outside the display screen such that the position of the virtualobject OBJ set outside the stereo loudspeakers of the informationprocessing apparatus 3 is the sound source localization position.Further, the vibrations to be imparted by the left actuator 373L and theright actuator 373R are adjusted so that the left and right vibrationintensities have a predetermined balance, and the user is caused toperceive as the vibration source the position of the virtual object OBJset outside the left actuator 373L and the right actuator 373R, byvisual sensation for indicating to the user that the virtual object OBJhas moved to outside the display screen and auditory sense forindicating the sound source localization position of the above 3D audioeffects to the user.

A description is given of the vibrations when, for example, after thevirtual object OBJ has moved from the virtual world outside the displayscreen of the display section 35 to the virtual world displayed withinthe display screen, the virtual object OBJ passes through the virtualworld within the display screen and moves to the virtual world outsidethe display screen. To make the description specific, as shown in FIG.9, a case is assumed where the virtual object OBJ moves from the virtualworld formed outside the display screen on the left side, passes throughthe virtual world displayed on the display screen, and moves to thevirtual world formed outside the display screen on the right side (thevirtual object OBJ moves along a path in the order of OBJ1, OBJ2, OBJ3,OBJ4, OBJ5, and OBJ6 shown in FIG. 9). It should be noted that thefollowing description is given on the assumption that the distancebetween the left actuator 373L and the right actuator 373R is X (if thedistance is almost the same as the width of the display screen of thedisplay section 35 in the left-right direction, the width is also X).

If the virtual object OBJ moves along the above movement path, theintensity of the vibration imparted by the left actuator 373L isstrongest at the time when the virtual object OBJ passes through theposition where the left actuator 373L is provided (i.e., the positionwhere the user holds the main body of the information processingapparatus 3 with the left hand, and a position near the left end of thedisplay screen of the display section 35) (at the time when the virtualobject OBJ passes through the position indicated by the virtual objectOBJ3) (the peak of the intensity of the vibration is defined as avibration intensity P). Then, when the virtual object OBJ passes throughthe position where the right actuator 373R is provided (i.e., theposition where the user holds the main body of the informationprocessing apparatus 3 with the right hand; a position near the rightend of the display screen of the display section 35; a position to theright, at the distance X, of the position where the intensity of thevibration is at the peak; and the position indicated by the virtualobject OBJ4), the intensity of the vibration imparted by the leftactuator 373L is ¼ of the vibration intensity P at the peak. Then, ifthe virtual object OBJ moves to the virtual world formed outside thedisplay screen on the right side and when the virtual object OBJ passesthrough a position (the position indicated by the virtual object OBJ5)outside the display screen further to the right, at the distance X, ofthe position indicated by the virtual object OBJ4, the intensity of thevibration imparted by the left actuator 373L is 1/16 of the vibrationintensity P at the peak. Further, when the virtual object OBJ passesthrough a position (the position indicated by the virtual object OBJ6)outside the display screen to the right, at a distance 2X, of theposition indicated by the virtual object OBJ4, the intensity of thevibration imparted by the left actuator 373L is 1/64 of the vibrationintensity P at the peak.

If the virtual object OBJ moves in the virtual world formed outside thedisplay screen on the left side and when the virtual object OBJ passesthrough a position (the position indicated by the virtual object OBJ2)outside the display screen to the left, at the distance X, of theposition where the intensity of the vibration is at the peak (theposition indicated by the virtual object OBJ3), the intensity of thevibration imparted by the left actuator 373L is ¼ of the vibrationintensity P at the peak. Further, when the virtual object OBJ passesthrough a position (the position indicated by the virtual object OBJ1)outside the display screen to the left, at the distance 2X, of theposition where the intensity of the vibration is at the peak, theintensity of the vibration imparted by the left actuator 373L is 1/16 ofthe vibration intensity Pat the peak. If the virtual object OBJ movesalong the above movement path, the intensity of the vibration impartedby the left actuator 373L is changed based on a function connectingthese points (a function defining a curve shown in FIG. 9). As is clearfrom FIG. 9, the intensity of the vibration imparted by the leftactuator 373L is changed based on a function of attenuating theintensity of the vibration symmetrically to the left and right of theposition where the intensity of the vibration is at the peak.

If, on the other hand, the virtual object OBJ moves along the abovemovement path, the intensity of the vibration imparted by the rightactuator 373R is strongest at the time when the virtual object OBJpasses through the position where the right actuator 373R is provided(i.e., the position where the user holds the main body of theinformation processing apparatus 3 with the right hand, and a positionnear the right end of the display screen of the display section 35) (atthe time when the virtual object OBJ passes through the positionindicated by the virtual object OBJ4) (the peak of the intensity of thevibration is defined as a vibration intensity P). Then, when the virtualobject OBJ passes through the position where the left actuator 373L isprovided (a position to the left, at a distance X, of the position wherethe intensity of the vibration is at the peak; a position near the leftend of the display screen of the display section 35; and the positionindicated by the virtual object OBJ3), the intensity of the vibrationimparted by the right actuator 373R is ¼ of the vibration intensity P atthe peak. Then, if the virtual object OBJ moves in the virtual worldformed outside the display screen on the left side and when the virtualobject OBJ passes through a position (the position indicated by thevirtual object OBJ2) outside the display screen further to the left, atthe distance X, of the position indicated by the virtual object OBJ3,the intensity of the vibration imparted by the right actuator 373R is1/16 of the vibration intensity P at the peak. Further, when the virtualobject OBJ passes through a position (the position indicated by thevirtual object OBJ1) outside the display screen to the left, at adistance 2X, of the position indicated by the virtual object OBJ3, theintensity of the vibration imparted by the right actuator 373R is 1/64of the vibration intensity P at the peak.

If the virtual object OBJ moves in the virtual world formed outside thedisplay screen on the right side and when the virtual object OBJ passesthrough a position (the position indicated by the virtual object OBJ5)outside the display screen to the right, at the distance X, of theposition where the intensity of the vibration is at the peak (theposition indicated by the virtual object OBJ4), the intensity of thevibration imparted by the right actuator 373R is ¼ of the vibrationintensity P at the peak. Further, when the virtual object OBJ passesthrough a position (the position indicated by the virtual object OBJ6)outside the display screen to the right, at the distance 2X, of theposition where the intensity of the vibration is at the peak, theintensity of the vibration imparted by the right actuator 373R is 1/16of the vibration intensity P at the peak. If the virtual object OBJmoves along the above movement path, also the intensity of the vibrationimparted by the right actuator 373R is changed based on a function ofconnecting these points (a function defining a curve shown in FIG. 9).Further, as is clear from FIG. 9, also the intensity of the vibrationimparted by the right actuator 373R is changed based on a function ofattenuating the intensity of the vibration symmetrically to the left andright of the position where the intensity of the vibration is at thepeak.

As described above, if the virtual object OBJ moves in the virtual worldformed outside the display screen, the intensities of the vibrationsimparted by the left actuator 373L and the right actuator 373R areadjusted to have a predetermined balance, thereby causing the user toperceive a position outside the display screen as the vibration source.In the example shown in FIG. 9, if the virtual object OBJ moves in thevirtual world formed outside the display screen on the right side, eachof the intensity of the vibration imparted by the left actuator 373L andthe intensity of the vibration imparted by the right actuator 373R iscontrolled by an attenuation function for obtaining a ratio of thesevibration intensities of 1:4. If, on the other hand, the virtual objectOBJ moves in the virtual world formed outside the display screen on theleft side, each of the intensity of the vibration imparted by the leftactuator 373L and the intensity of the vibration imparted by the rightactuator 373R is controlled by an attenuation function for obtaining aratio of these vibration intensities of 4:1.

It should be noted that the balance between the intensities of thevibrations imparted by the left actuator 373L and the right actuator373R when the user is caused to perceive a position outside the displayscreen as the vibration source does not need to be the above ratio.Alternatively, the above ratio may be appropriately set in accordancewith the absolute values of the intensities of the vibrations to beimparted, the structure of the main body of the information processingapparatus 3, the absolute value of the distance X, the distance betweenthe display screen and the position to be perceived as the vibrationsource, and the like. Yet alternatively, the balance may be controlledso that one of the intensities of the vibrations imparted by the leftactuator 373L and the right actuator 373R is 0.

Further, in FIG. 9, an example has been used where the intensities ofthe vibrations imparted by the left actuator 373L and the right actuator373R are changed based on a function connecting the above points (anattenuation function defining a curve shown in FIG. 9). Alternatively,each of the intensities of the vibrations may be changed based onanother attenuation function. For example, each of the intensities ofthe vibrations imparted by the left actuator 373L and the right actuator373R may be a linear function or a quadratic function of attenuating theintensity of the vibration symmetrically to the left and right of theposition where the intensity of the vibration is at the peak, and may bechanged based on a high-dimensional function of three or more dimensionsor another attenuation function. Further, control may be performed sothat the vibration intensity P at the peak of the vibration imparted byeach of the left actuator 373L and the right actuator 373R is generatednot only at the above peak position but also when the virtual object OBJis moving within a certain range from the peak position. In this case,the range where a vibration having the vibration intensity P at the peakis imparted is set to a certain range. Further, a function ofattenuating each of the intensities of the vibrations imparted by theleft actuator 373L and the right actuator 373R from the peak positionmay not be a function of attenuating the intensity of the vibrationsymmetrically to the left and right of the peak position.

Further, an example has been described above where the intensities ofthe vibrations imparted by the left actuator 373L and the right actuator373R are attenuated. Alternatively, the sounds to be output from theleft and right loudspeakers (e.g., the sound volumes) may be controlledusing a similar attenuation function.

Further, in the first exemplary game and the second exemplary gamedescribed above, the position to be perceived as the vibration source isset in accordance with the position of each of the player character PCand the virtual object OBJ moving in the virtual world. Alternatively,the position to be perceived as the vibration source in the exemplaryembodiment does not need to be the position of an image of which themovement is displayed. For example, the vibrations to be imparted by theleft actuator 373L and the right actuator 373R may be controlled so thata virtual object fixedly displayed on the display screen of the displaysection 35 is the vibration source. Further, if an image obtained bycapturing the real world is displayed on the display screen of thedisplay section 35, the vibrations to be imparted by the left actuator373L and the right actuator 373R may be controlled so that the positionof a captured object in the real world that appears in the image is thevibration source.

Next, a detailed description is given of the processing performed by theinformation processing apparatus 3. First, with reference to FIG. 10,main data used in the processing is described. FIG. 10 is a diagramshowing examples of main data and programs stored in the storage section32 of the information processing apparatus 3. It should be noted that inthe following exemplary processing, a description is given using theinformation processing when the game processing of the above secondexemplary game is performed.

As shown in FIG. 10, the following are stored in the data storage areaof the storage section 32: operation data Da; object position data Db;vibration setting information data Dc; sound setting information dataDd; vibration control signal data De; sound control signal data Df;vibration data Dg; sound data Dh; display image data Di; and the like.It should be noted that the storage section 32 may store, as well as thedata shown in FIG. 10, data and the like necessary for the processing,such as data used in an application to be executed. Further, in theprogram storage area of the storage section 32, various programs Paincluded in the information processing program are stored. For example,the various programs Pa include a vibration generation program forgenerating vibration control signals to impart vibrations to theinformation processing apparatus 3, a sound generation program forgenerating sound control signals to output sounds from the informationprocessing apparatus 3, an image display program for displaying an imageon the display section 35, and the like.

The operation data Da is data representing the content of the operationperformed on the input section 34 and includes, for example, datarepresenting the touch position of a touch operation on the touch panel341. It should be noted that if the input section 34 includes a sensorfor detecting the orientation and the motion of the main body of theinformation processing apparatus 3, the operation data Da may includedata for calculating the orientation and the motion of the main body ofthe information processing apparatus 3 (e.g., data representing theacceleration generated in the main body of the information processingapparatus 3 and data representing the angular velocity of the main bodyof the information processing apparatus 3).

The object position data Db is data representing the position of avirtual object OBJ moving in a virtual world (see FIGS. 4 and 8).

The vibration setting information data Dc includes vibration type dataDc1, vibration source position data Dc2, and the like. The vibrationtype data Dc1 is data representing the type of vibrations to be impartedto the information processing apparatus 3. The vibration source positiondata Dc2 is data representing the position to be perceived as thevibration source by the user of the information processing apparatus 3.

The sound setting information data Dd includes sound source type dataDd1, sound source localization position data Dd2, and the like. Thesound source type data Dd1 is data representing the type of sounds to beoutput from the information processing apparatus 3. The sound sourcelocalization position data Dd2 is data representing the sound sourcelocalization position of sounds to be output from the informationprocessing apparatus 3.

The vibration control signal data De is data representing vibrationcontrol signals to be output from the control section 31 to thevibration generation section 37 (the left vibration control signal CSLand the right vibration control signal CSR; see FIG. 3). The soundcontrol signal data Df is data representing sound control signals to beoutput from the control section 31 to the sound output section 36 (theleft sound control signal and the right sound control signal).

The vibration data Dg is data prepared in advance for generatingvibration control signals and is stored for each type of vibration to beimparted to the main body of the information processing apparatus 3(e.g., for each virtual object for which vibrations are generated). Thesound data Dh is data prepared in advance for generating sound controlsignals and is stored for each type of sound to be output from theinformation processing apparatus 3 (e.g., for each virtual object forwhich sounds are produced or each type of BGM).

The display image data Di is data for generating an image of eachvirtual object such as the virtual object OBJ, a background image, andthe like and displaying the generated images on the display section 35.

Next, with reference to FIG. 11, a detailed description is given of thegame processing, which is an example of the information processingperformed by the information processing apparatus 3. It should be notedthat FIG. 11 is a flow chart showing an example of the game processingperformed by the information processing apparatus 3. Here, in the flowchart shown in FIG. 11, a description is given mainly of, in theprocessing performed by the information processing apparatus 3, theprocess of outputting vibrations and sounds corresponding to themovement of the virtual object OBJ in the virtual world in the secondexemplary game. The detailed descriptions of other processes notdirectly related to these processes are omitted. Further, in FIG. 11,all of the steps performed by the control section 31 are abbreviated as“S”.

The CPU of the control section 31 initializes a memory and the like ofthe storage section 32 and loads the information processing program fromthe program storage section 33 into the memory. Then, the CPU starts theexecution of the information processing program. The flow chart shown inFIG. 11 is a flow chart showing the processing performed after the aboveprocesses are completed.

It should be noted that the processes of all of the steps in the flowchart shown in FIG. 11 are merely illustrative. Thus, the processingorder of the steps may be changed, or another process may be performedin addition to and/or instead of the processes of all of the steps, solong as similar results are obtained. Further, in the exemplaryembodiment, descriptions are given on the assumption that the controlsection 31 (the CPU) performs the processes of all of the steps in theflow chart. Alternatively, the CPU may perform the processes of some ofthe steps in the flow chart, and a processor or a dedicated circuitother than the CPU may perform the processes of the other steps. Yetalternatively, a processor or a dedicated circuit other than the CPU mayperform the processes of all of the steps in the flow chart.

Referring to FIG. 11, the control section 31 performs initialization(step 41), and proceeds to the subsequent step. For example, the controlsection 31 constructs a virtual world to be displayed on the displaysection 35 and initializes parameters. As an example, the controlsection 31 places the virtual object OBJ at an initial position in thevirtual world and sets the object position data Db. Further, the controlsection 31 sets the display range to be displayed on the display screenof the display section 35 for the virtual world.

Next, the control section 31 acquires operation data from the inputsection 34 and updates the operation data Da (step 42), and theprocessing proceeds to the next step.

Next, the control section 31 performs the process of causing the virtualobject OBJ to move in the virtual world (step 43), and the processingproceeds to the next step. For example, the control section 31 causesthe virtual object OBJ to move at a moving speed determined in advancealong a movement path determined in advance in the virtual world andupdates the object position data Db using the position of the virtualobject OBJ after the movement. As another example, if the controlsection 31 causes the virtual object OBJ to move in accordance with anoperation on the input section 34 (including the operation of moving ortilting the main body of the information processing apparatus 3), thecontrol section 31 causes the virtual object OBJ to move in the virtualworld in accordance with the operation data acquired in the above step42 and updates the object position data Db using the position of thevirtual object OBJ after the movement.

Next, the control section 31 sets the type of vibrations and theposition of the vibration source (step 44), and the processing proceedsto the next step. For example, based on the vibration generation programand the type of the virtual object OBJ placed in the virtual world, thecontrol section 31 sets the type of vibrations when the virtual objectOBJ moves. Then, the control section 31 updates the vibration type dataDc1 using data representing the type of vibrations. Further, based onthe vibration generation program, the control section 31 sets theposition of the vibration source such that the position of the virtualobject OBJ indicated by the object position data Db is perceived as thevibration source by the user. Then, the control section 31 updates thevibration source position data Dc2 using data representing the setposition.

Next, based on vibration setting information, the control section 31sets vibration control signals (step 45), and the processing proceeds tothe next step. For example, based on the vibration generation programand the vibration setting information data Dc (the vibration type dataDc1 and the vibration source position data Dc2), the control section 31generates vibration control signals (the left vibration control signalCSL and the right vibration control signal CSR to be output to thevibration generation section 37; see FIG. 3) using vibration data readfrom the vibration data Dg and stores the vibration control signals inthe vibration control signal data De. Specifically, the control section31 reads data from the vibration data Dg and generates the leftvibration control signal CSL and the right vibration control signal CSRcorresponding to the vibrations of the type indicated by the vibrationtype data Dc1, so that the vibrations of the type indicated by thevibration type data Dc1 are imparted to the main body of the informationprocessing apparatus 3, and the position indicated by the vibrationsource position data Dc2 is perceived as the vibration source of thevibrations.

Next, the control section 31 sets the type of sound sources and theposition where the sound sources are localized (step 46), and theprocessing proceeds to the next step. For example, based on the soundgeneration program and the type of the virtual object OBJ placed in thevirtual world, the control section 31 sets the type of sounds when thevirtual object OBJ moves. Then, the control section 31 updates the soundsource type data Dd1 using data representing the type of sounds.Further, based on the sound generation program, the control section 31sets the sound source localization position such that the position ofthe virtual object OBJ indicated by the object position data Db is thesound source localization position of the sounds. Then, the controlsection 31 updates the sound source localization position data Dd2 usingdata representing the sound source localization position.

Next, based on sound setting information, the control section 31 setssound control signals (step 47), and the processing proceeds to the nextstep. For example, based on the sound generation program and the soundsetting information data Dd (the sound source type data Dd1 and thesound source localization position data Dd2), the control section 31generates sound control signals (the left sound control signal and theright sound control signal to be output to the sound output section 36)and stores the sound control signals in the sound control signal dataDf. Specifically, the control section 31 reads data from the sound dataDh and generates the left sound control signal and the right soundcontrol signal, so that the sounds of the type indicated by the soundsource type data Dd1 are output from the stereo loudspeakers of theinformation processing apparatus 3, and the position indicated by thesound source localization position data Dd2 is the sound localizationposition.

Next, the control section 31 performs a display control process (step48), and the processing proceeds to the next step. For example, based onan image generation program and the object position data Db, the controlsection 31 performs the process of generating an image of the virtualworld in which the virtual object OBJ is placed, and displaying on thedisplay section 35 the image of the virtual world in the set displayrange.

Next, the control section 31 performs a control signal output process(step 49), and the processing proceeds to the next step. For example,the control section 31 outputs to the vibration generation section 37the left vibration control signal CSL and the right vibration controlsignal CSR indicated by the vibration control signal data De.Consequently, the vibration generation section 37 generates a vibrationcorresponding to the left vibration control signal CSL from the leftactuator 373L and generates a vibration corresponding to the rightvibration control signal CSR from the right actuator 373R. Further, thecontrol section 31 outputs to the sound output section 36 the left soundcontrol signal and the right sound control signal indicated by the soundcontrol signal data Df. Consequently, the sound output section 36outputs a sound corresponding to the left sound control signal from theleft loudspeaker and outputs a sound corresponding to the right soundcontrol signal from the right loudspeaker.

Next, the control section 31 determines whether or not the gameprocessing is to be ended (step 50). Examples of conditions for endingthe game processing include: the satisfaction of the condition underwhich the game processing is ended; and the fact that the user hasperformed the operation of ending the game processing. If the gameprocessing is not to be ended, the control section 31 returns to theabove step 42 and repeats the process thereof. If the game processing isto be ended, the control section 31 ends the processing indicated by theflow chart.

As described above, in the game processing according to the aboveexemplary embodiment, if the virtual object OBJ moves in the virtualworld, vibrations in a wide band imparted by the left actuator 373L andthe right actuator 373R are imparted to the information processingapparatus 3, whereby it is possible to impart vibrations rich in varietyto the user of the apparatus. Further, in the game processing accordingto the above exemplary embodiment, it is possible to cause the user toperceive as the vibration source the position where the virtual objectOBJ is placed in the virtual world. Here, in the game processingaccording to the above exemplary embodiment, sounds in a wide band whenthe virtual object OBJ moves are output such that the position where thevirtual object OBJ is placed in the virtual world is the sound sourcelocalization position of the sounds output from the informationprocessing apparatus 3. Further, if the virtual object OBJ is placed inthe virtual world displayed on the display screen of the display section35, the position where the virtual object OBJ is placed is displayed onthe display screen of the display section 35. As described above, theposition of the virtual object OBJ is presented using tactile sensationbased on vibrations in a wide band, auditory sense based on sounds in awide band, and visual sensation based on the display of a moving objecton the display screen, whereby it is possible to impart unconventionalbodily sensation rich in reality to the user.

Further, even if the virtual object OBJ is moving in the virtual worldset outside the display screen, vibrations to be imparted by the leftactuator 373L and the right actuator 373R are adjusted to have apredetermined balance, and the sound source localization position is setto the position of the virtual object OBJ outside the display screenwhile indicating to the user that the virtual object OBJ has moved tooutside the display screen. Thus, it is also possible to cause the userto perceive as the vibration source even the position of the virtualobject OBJ set outside the left actuator 373L and the right actuator373R.

It should be noted that the above descriptions are given using theexample where the information processing apparatus 3 performsinformation processing (game processing). Alternatively, anotherapparatus may perform at least some of the processing steps in theinformation processing. For example, if the information processingapparatus 3 is further configured to communicate with another apparatus(e.g., another server, another game apparatus, or another mobileterminal), the other apparatus may cooperate to perform the processingsteps of the information processing. As an example, another apparatusmay perform at least one of the virtual world image generation process,the vibration control signal generation process, and the sound controlsignal generation process, and the information processing apparatus 3may acquire image data and control signals indicating the result of theprocess. Another apparatus may thus perform at least some of theprocessing steps in the information processing, thereby enablingprocessing similar to the above information processing. Further, theabove information processing can be performed by a processor or thecooperation of a plurality of processors, the processor or the pluralityof processors included in an information processing system including atleast one information processing apparatus. Further, in the aboveexemplary embodiment, the processing indicated in the above flow chartis performed by the control section 31 of the information processingapparatus 3 executing a predetermined game program. Alternatively, apart or all of the information processing indicated in the flow chartmay be performed by a dedicated circuit included in the informationprocessing apparatus 3.

Here, the above variations make it possible to achieve the exemplaryembodiment also by a system form such as so-called cloud computing, or asystem form such as a distributed wide area network or a local areanetwork. For example, in a system form such as a distributed local areanetwork, it is possible to execute the information processing between astationary information processing apparatus (a stationary gameapparatus) and a handheld information processing apparatus (a handheldgame apparatus) by the cooperation of the apparatuses. It should benoted that, in these system forms, there is no particular limitation onwhich apparatus performs the process of each step of the aboveinformation processing. Thus, it goes without saying that it is possibleto achieve the exemplary embodiment by sharing the processing in anymanner.

In addition, the processing orders, the setting values, the conditionsused in the determinations, and the like that are used in the aboveinformation processing are merely illustrative. Thus, it goes withoutsaying that the exemplary embodiment can be achieved also with otherorders, other values, and other conditions. Further, the shapes, thenumber, the placement positions, the functions, and the like of thecomponents used by the above information processing apparatus are merelyillustrative, and may be other shapes, number, and placement positions.It goes without saying that the exemplary embodiment can be achieved bythe information processing apparatus having other functions. As anexample, three or more actuators may impart vibrations to theinformation processing apparatus, or three or more loudspeakers mayoutput sounds from the information processing apparatus. Alternatively,the information processing apparatus may include a plurality of displaysections. Further, in the above description, a handheld apparatus (e.g.,a tablet terminal) has been used an example of the informationprocessing apparatus 3. Alternatively, the information processingapparatus 3 may be a portable apparatus larger than a handheldapparatus. Here, a portable apparatus is an apparatus that allows themovement of the main body of the apparatus when the apparatus is used,or allows a change in the orientation of the main body of the apparatuswhen the apparatus is used, or allows the carrying around of the mainbody of the apparatus, and is a concept including the above handheldapparatus.

In addition, the information processing program may be supplied to theinformation processing apparatus 3 not only through an external storagemedium such as the external memory 45, but also through a wired orwireless communication link. Further, the information processing programmay be stored in advance in a non-volatile storage device included inthe information processing apparatus 3. It should be noted that examplesof an information storage medium having stored therein the informationprocessing program may include CD-ROMs, DVDs, optical disk storage mediasimilar to these, flexible disks, hard disks, magneto-optical disks, andmagnetic tapes, as well as non-volatile memories. Alternatively, aninformation storage medium having stored therein the informationprocessing program may be a volatile memory for storing the informationprocessing program. It can be said that such a storage medium is astorage medium readable by a computer or the like. For example, it ispossible to provide the above various functions by causing a computer orthe like to load a game program from the storage medium and execute it.

While some exemplary systems, exemplary methods, exemplary devices, andexemplary apparatuses have been described in detail above, the abovedescriptions are merely illustrative in all respects, and do not limitthe scope of the systems, the methods, the devices, and the apparatuses.It goes without saying that the systems, the methods, the devices, andthe apparatuses can be improved and modified in various manners withoutdeparting the spirit and scope of the appended claims. It is understoodthat the scope of the systems, the methods, the devices, and theapparatuses should be interpreted only by the scope of the appendedclaims. Further, it is understood that the specific descriptions of theexemplary embodiment enable a person skilled in the art to carry out anequivalent scope on the basis of the descriptions of the exemplaryembodiment and general technical knowledge. It should be understoodthat, when used in the specification, the components and the likedescribed in the singular with the word “a” or “an” preceding them donot exclude the plurals of the components. Furthermore, it should beunderstood that, unless otherwise stated, the terms used in thespecification are used in their common meanings in the field. Thus,unless otherwise defined, all the jargons and the technical terms usedin the specification have the same meanings as those generallyunderstood by a person skilled in the art in the field of the exemplaryembodiment. If there is a conflict, the specification (includingdefinitions) takes precedence.

As described above, the exemplary embodiment is useful as, for example,an information processing apparatus, an information processing program,an information processing system, an information processing method, andthe like in order, for example, to impart vibrations rich in variety toa user.

What is claimed is:
 1. A system comprising: a display having a displayarea; a pair of vibrators; and one or more processors operativelycoupled to the display and the pair of vibrators, the one or moreprocessors configured to: generate stereo sound signals; generatevibration signals for controlling the pair of vibrators to vibrate, thevibration signals determining a position to perceive a vibration source;and generate image signals to control an image to be displayed on thedisplay, wherein: in controlling the image, the one or more processorscontrol display on the display of at least one object moving from acorresponding position in a virtual world displayed on the display to acorresponding position in the virtual world outside the display areaand/or moving from the corresponding position in the virtual worldoutside the display area to the corresponding position in the virtualworld displayed on the display, and in generating the vibration signals,even if the object is placed at the corresponding position in thevirtual world outside the display area, the one or more processorscontrol a position of a vibration source to be perceived based on thevibrations of the vibrators in accordance with placement position of theobject.
 2. The system of claim 1 wherein the one or more processorsgenerate vibration signals to maximize vibration amplitude at theinstant when the object moves from the display area to outside thedisplay area.
 3. The system of claim 1 wherein the one or moreprocessors generate vibration signals to maximize vibration amplitude atthe instant when the object moves from outside the display area towithin the display area.
 4. The system of claim 1 wherein the one ormore processors generate vibration signals that are configured toindicate a perceived phantom single position when one of the pluralvibrators is held by a first hand of a user and another of the pluralvibrators is held by a second hand of the user.
 5. The system of claim 1wherein the one or more processors generate vibration signals that havea predetermined balance when the object is displayed outside of thedisplay area, thereby providing a simulated vibration source positionoutside the display area.
 6. The system of claim 1 wherein the one ormore processors are configured to control the ratio of one of thevibration signals relative to another of the vibration signals.
 7. Thesystem of claim 1 wherein the one or more processors are configured tocontrol the relative amplitudes of the vibration signals as a linearfunction.
 8. The system of claim 1 wherein the one or more processorsare configured to control the relative amplitudes of the vibrationsignals as a quadratic function.
 9. The system of claim 1 wherein theone or more processors are configured to control the relative amplitudesof the vibration signals to simulate a vibration source that is outsideof the display area.
 10. A system comprising: a display having a displayarea; a first handheld vibrator; a second handheld vibrator; and one ormore processors operatively coupled to the display and the first andsecond vibrators, the one or more processors configured to: generateimage signals to control an image to be displayed on the display,wherein in controlling the image, the one or more processors controldisplay on the display of at least one object moving from acorresponding position in a virtual world displayed on the display to acorresponding position in the virtual world outside the display areaand/or moving from the corresponding position in the virtual worldoutside the display area to the corresponding position in the virtualworld displayed on the display, and generate a first vibration signalfor controlling the first vibrator and a second vibration control signalfor controlling the second vibrator, the first and second vibrationsignals being configured to control the first and second vibrators togenerate vibration that is perceived as originating from a singlevibration source outside the display area.
 11. The system of claim 10wherein the one or more processors generate vibration signals tomaximize vibration amplitude at the instant when the object moves fromthe display area to outside the display area.
 12. The system of claim 10wherein the one or more processors generate vibration signals tomaximize vibration amplitude at the instant when the object moves fromoutside the display area to within the display area.
 13. The system ofclaim 10 wherein the one or more processors generate vibration signalsthat are configured to indicate a perceived phantom single position whenone of the plural vibrators is held by a first hand of a user andanother of the plural vibrators is held by a second hand of the user.14. The system of claim 10 wherein the one or more processors generatevibration signals that have a predetermined balance when the object isdisplayed outside of the display area, thereby providing a simulatedvibration source position outside the display area.
 15. The system ofclaim 10 wherein the one or more processors are configured to controlthe ratio of one of the vibration signals relative to another of thevibration signals.
 16. The system of claim 10 wherein the one or moreprocessors are configured to control the relative amplitudes of thevibration signals as a linear function.
 17. The system of claim 10wherein the one or more processors are configured to control therelative amplitudes of the vibration signals as a quadratic function.18. The system of claim 10 wherein the one or more processors areconfigured to control the relative amplitudes of the vibration signalsto simulate a vibration source that is outside of the display area. 19.A system comprising: a display having a display area; a first vibratorfor applying vibrations to a first hand; a second vibrator for applyingvibrations to a second hand; and a processor operatively coupled to thedisplay and the first and second vibrators, the processor configured togenerate a first vibration signal for controlling the first vibrator anda second vibration control signal for controlling the second vibrator,the first and second vibration signals being configured to togethercontrol the first and second vibrators to generate vibration that isperceived through the first and second hands as originating from avibration source outside the display area.
 20. The system of claim 19wherein the one or more processors generate vibration signals that areconfigured to indicate a perceived phantom single position when one ofthe plural vibrators is held by the first hand of a user and another ofthe plural vibrators is held by the second hand of the user.